EE 377/STATS 311 Project Report: Learning from Different Domains

Most machine learning algorithms are theoretically designed and practically applied with the consideration that the training data and test data arise from a common distribution. In several applications in natural language processing ranging from information extraction or sentiment analysis, classification algorithms are trained using a limited set of documents and applied generally to works spanning different genres. Also, in genetics, most experiments are done and predictive methods developed for people from a particular geography but this is broadly applied to all humans whose genetic makeup arises from a different distribution. It is of interest to learn how efficiently algorithms trained on samples from a source distribution perform on test data arising from a target distribution. In [1], the authors investigate this problem of domain adaptation. The authors first consider the problem of binary classification trained on labeled data from a source distribution. Classification error bounds are presented in terms of source domain error and divergence measures between the two distributions. Unlabeled data from either domain is then leveraged to estimate the divergence between domains to present a second bound. The third question the authors answer is learning with different amounts of labeled target and source data and they do this via a hypothesis that minimizes a convex combination of the error. We aim to extend the results of the paper in a few directions. In Section 2, we introduce the problem setup. We then present a simple upper bound on the generalization error with different domains. The paper provides this result for only binary function with 0-1 loss. We extend it to arbitrary functions and a large class of loss functions. Seeking to make this bound tighter, we define a classifier based distance for a larger class of hypothesis and loss functions. We now extend the results by giving guarantees on an algorithm that seeks to minimize a convex combination of training error and test error. We attempt to use this bound to show what amount of regularization is appropriate. In Section 3, we present an alternate view of looking at the correct amount of regularization. Finally, we validate some of the theoretical results with simulations in Section 4. Conclusions are presented in Section 5.